to this type.
There are a lot of discussions about highly available clusters, so I will not go into detail here.
3. High-performance computing cluster
3.1 What is a high-performance computing cluster
In short, High-Performance Computing is a branch of computer science. It
Cloud computing and cloud Database Cluster architect, cloud database cluster architecture
Action to fulfill your dreams
Course
Course Overview
Course Module
Class requirements
LinuxSome courses
ULE-LinuxBasic
Linux environment and permission management
LINUX beginner and advanced
Linux system configuration and Network
This article is a summary and application of some experience in cluster resource management in the development of real-time computing platform by the micro-BO operation and data Platform (DIP), which mainly focuses on the following issues:
How does heterogeneous resources integrate?
How do real-time computing the physical resources between applicatio
/hosts in each node and write the correspondence between the host name and ip address of each node in the cluster.
3.ConfigurationNFSShared File System
Distributed Parallel Computing generally requires that the application software environment of each node server be consistent with the working directory environment. It is especially troublesome to configure it o
server-side development require a lot of manpower to deploy and maintain various software, I feel that this is supposed to free up more labor.
First of all, the HA cluster environment in this article is RHEL6.4 Virtual Machine (KVM). The configurations of each virtual machine are identical, except that the IP address, MAC address, and HOSTNAME are different. The vm host is Ubuntu14.04 Desktop on my laptop.
:
HOSTNAME = node1
Modify the file/etc/hosts in each node and write the correspondence between the host name and ip address of each node in the cluster.
3. Configure the NFS Shared File System
Distributed Parallel Computing generally requires that the application software environment of each node server be consistent with the working directory environment. It is
Nbsp; 1. the installation and installation process for linux is not mentioned much. Note that during installation, if the cluster is not connected to the outside world, you can select rsh as a trusted service regardless of security considerations. Be sure to install the corresponding software package; if you need to connect to the outside world, you should select ssh as a trusted service for security purpos
relationship between the host name and IP of each node in the cluster.3. Configuring the NFS shared file systemDistributed parallel computing generally requires the application software environment and working directory environment of each node server to be consistent, which is particularly troublesome if it is configured on each node. Therefore, the use of NFS
provides assistance and relative protection for the whole construction, and in this management process, cloud computing itself can use different kinds of requirements under different circumstances to complete the construction of multiple computer cluster system, its function, significance and value are self-evident. "2"with the use of cloud computing technology,
directory;
2. Get the step flow: Under the newly established directory, go to the INSTALL and README and other related files (important steps!);
3. Dependency Properties Software Installation: According to the content of Install/readme and install some dependent software (not necessary);
4. Establish makefile: detect the working environment with the Automatic detection program (configure or config) and est
starting it.Summary of commands in hadoopThis part of content can be understood through the help and introduction of the command. I mainly focus on introducing a few of the commands I use. The hadoop DFS command is followed by a parameter for HDFS operations, which is similar to the Linux Command, for example:
Hadoop DFS-ls is to view the content in the/usr/root directory. By default, if no path is specified, this is the current user path;
Hadoop DFS-rmr xxx is used to delete directories. It
Iptablesiptables-restore iptables Restores the last saved iptables State iptables-save >/etc/sysconfig/ iptables can also be saved in this way. EL7 firewall firewalld1. editing mode firewall- Config graphical tool 2. Command line: firewall- cmd3 . Status control systemctl Restart Firewalld stop Kernel: Module 1. module: Provide some functions, need to load up, do not need to uninstall, can have a personalized control of the server. 2. View the modules that have been loaded lsmod3. Where
Wang Jialin's in-depth case-driven practice of cloud computing distributed Big Data hadoop in July 6-7 in Shanghai
Wang Jialin Lecture 4HadoopGraphic and text training course: Build a true practiceHadoopDistributed Cluster EnvironmentHadoopThe specific solution steps are as follows:
Step 1: QueryHadoopTo see the cause of the error;
Step 2: Stop the cluster;
uses the RPM method, the Debian-like system uses the DEB method, the BSD-like system uses the TAR method)3Test your packaged application software installation package on a test machine or test machine4) Deploy the package to the system that needs it5) Verify its health on the deployment server6if necessary, you can restore the previous system structure and configuration2. Create a standard document • File system list • Third-party
SSH noden.
4. Modify (or create) a/etc/hosts.equiv fileFill in all the machine names that you allow to access this computer for MPI computing, one line for a machine name. This step is to allow the node to delegate to the other nodes.For example, Node1 is the machine used to start MPI cluster computing, and the other nodes are involved in the computation, as in
public_content_rw_t pub-- GT; Note the path/var/ftp/pub restorecon-r-v pub--for/var/ftp/pub setsebool-p allow_ftpd_anon_write. Black and White List blacklist:/etc/vsftpd/ftpusers in the main configuration file there is a row parameter: userlist_enable=Yes if the parameter is yes, the/etc/vsftpd/user_ List is a blacklist. If the parameter is no, the/etc/vsftpd/user_list is the whitelist. If there is no configuration for the row, the default parameter is No. Man 5 vsftpd.conf 0 Basic Learning
Tags: too inux col iptables file mod soft rom BaseURL1. Check for Hyper-threadingTo view the number of threads:grep siblings/proc/cpuinfo |uniqTo view the number of physical cores: grep cores/proc/cpuinfo |uniq2. Check the operating system versionCat/etc/redhat-releaseLsb_release-a3. Check the file systemDf-th4. Turn off SELinuxEdit Profile: Vi/etc/sysconfig/selinuxConfirmation file is: Confirm selinux=disabled5. Turn off the firewallPurge Firewall rule: iptables-fTurn off firewall/etc/init.d/ip
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.